{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RkT_im5zP3GD"
      },
      "source": [
        "[<img src='https://github.com/jeshraghian/snntorch/blob/master/docs/_static/img/snntorch_alpha_w.png?raw=true' width=\"400\">](https://github.com/jeshraghian/snntorch/)\n",
        "\n",
        "# Regression with SNNs: Part II\n",
        "## Regression-based Classification with Recurrent Leaky Integrate-and-Fire Neurons\n",
        "## By Alexander Henkes (https://orcid.org/0000-0003-4615-9271) and Jason K. Eshraghian (www.ncg.ucsc.edu)\n",
        "\n",
        "\n",
        "<a href=\"https://colab.research.google.com/github/jeshraghian/snntorch/blob/master/examples/tutorial_regression_2.ipynb\">\n",
        "  <img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/>\n",
        "</a>\n",
        "\n",
        "[<img src='https://github.com/jeshraghian/snntorch/blob/master/docs/_static/img/GitHub-Mark-Light-120px-plus.png?raw=true' width=\"28\">](https://github.com/jeshraghian/snntorch/) [<img src='https://github.com/jeshraghian/snntorch/blob/master/docs/_static/img/GitHub_Logo_White.png?raw=true' width=\"80\">](https://github.com/jeshraghian/snntorch/)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4blpfg4y44uO"
      },
      "source": [
        "This tutorial is based on the following papers on nonlinear regression and spiking neural networks. If you find these resources or code useful in your work, please consider citing the following sources:\n",
        "\n",
        "> <cite> [Alexander Henkes, Jason K. Eshraghian, and Henning Wessels. “Spiking neural networks for nonlinear regression\", arXiv preprint arXiv:2210.03515, October 2022.](https://arxiv.org/abs/2210.03515) </cite>\n",
        "\n",
        "> <cite> [Jason K. Eshraghian, Max Ward, Emre Neftci, Xinxin Wang, Gregor Lenz, Girish Dwivedi, Mohammed Bennamoun, Doo Seok Jeong, and Wei D. Lu. \"Training Spiking Neural Networks Using Lessons From Deep Learning\". Proceedings of the IEEE, 111(9) September 2023.](https://ieeexplore.ieee.org/abstract/document/10242251) </cite>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lnF_PEo5obYv",
        "pycharm": {
          "name": "#%% md\n"
        }
      },
      "source": [
        "In the regression tutorial series, you will learn how to use snnTorch to perform regression using a variety of spiking neuron models, including:\n",
        "\n",
        "* Leaky Integrate-and-Fire (LIF) Neurons\n",
        "* Recurrent LIF Neurons\n",
        "* Spiking LSTMs\n",
        "\n",
        "An overview of the regression tutorial series:\n",
        "\n",
        "* Part I will train the membrane potential of a LIF neuron to follow a given trajectory over time.\n",
        "* Part II (this tutorial) will use LIF neurons with recurrent feedback to perform classification using regression-based loss functions\n",
        "* Part III will use a more complex spiking LSTM network instead to train the firing time of a neuron.\n",
        "\n",
        "If running in Google Colab:\n",
        "* You may connect to GPU by checking `Runtime` > `Change runtime type` > `Hardware accelerator: GPU`\n",
        "* Next, install the latest PyPi distribution of snnTorch and Tonic by clicking into the following cell and pressing `Shift+Enter`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "XzIk6vePP3GE"
      },
      "outputs": [],
      "source": [
        "!pip install snntorch --quiet"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "wFh5v0X9P3GF"
      },
      "outputs": [],
      "source": [
        "# imports\n",
        "import snntorch as snn\n",
        "from snntorch import surrogate\n",
        "from snntorch import functional as SF\n",
        "from snntorch import utils\n",
        "\n",
        "import torch\n",
        "import torch.nn as nn\n",
        "from torch.utils.data import DataLoader\n",
        "from torchvision import datasets, transforms\n",
        "import torch.nn.functional as F\n",
        "\n",
        "import matplotlib.pyplot as plt\n",
        "import numpy as np\n",
        "import itertools\n",
        "import tqdm"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6OWbvSEAP3GF"
      },
      "source": [
        "# 1. Classification as Regression"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "KkAuiz-FP3GF"
      },
      "source": [
        "In conventional deep learning, we often calculate the Cross Entropy Loss to train a network to do classification. The output neuron with the highest activation is thought of as the predicted class.\n",
        "\n",
        "In spiking neural nets, this may be interpreted as the class that fires the most spikes. I.e., apply cross entropy to the total spike count (or firing frequency). The effect of this is that the predicted class will be maximized, while other classes aim to be suppressed.\n",
        "\n",
        "The brain does not quite work like this. SNNs are sparsely activated, and while approaching SNNs with this deep learning attitude may lead to optimal accuracy, it's important not to 'overfit' too much to what the deep learning folk are doing. After all, we use spikes to achieve better power efficiency. Good power efficiency relies on sparse spiking activity.\n",
        "\n",
        "In other words, training bio-inspired SNNs using deep learning tricks does not lead to brain-like activity.\n",
        "\n",
        "So what can we do? \n",
        "\n",
        "We will focus on recasting classification problems into regression tasks. This is done by training the predicted neuron to fire a given number of times, while incorrect neurons are trained to still fire a given number of times, albeit less frequently.\n",
        "\n",
        "This contrasts with cross-entropy which would try to drive the correct class to fire at *all* time steps, and incorrect classes to not fire at all.\n",
        "\n",
        "As with the previous tutorial, we can use the mean-square error to achieve this. Recall the form of the mean-square error loss:\n",
        "\n",
        "$$\\mathcal{L}_{MSE} = \\frac{1}{n}\\sum_{i=1}^n(y_i-\\hat{y_i})^2$$\n",
        "\n",
        "where $y$ is the target and $\\hat{y}$ is the predicted value. \n",
        "\n",
        "To apply MSE to the spike count, assume we have $n$ output neurons in a classification problem, where $n$ is the number of possible classes. $\\hat{y}_i$ is now the total number of spikes the $i^{th}$ output neuron emits over the full simulation runtime.\n",
        "\n",
        "Given that we have $n$ neurons, this means that $y$ and $\\hat{y}$ must be vectors with $n$ elements, and our loss will sum the independent MSE losses of each neuron."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2LmU7QS4P3GF"
      },
      "source": [
        "## 1.2 A Theoretical Example\n",
        "\n",
        "Consider a simulation of 10 time steps. Say we wish for the correct neuron class to fire 8 times, and the incorrect classes to fire 2 times. Assume $y_1$ is the correct class:\n",
        "\n",
        "$$ y = \\begin{bmatrix} 8 \\\\ 2 \\\\ \\vdots \\\\ 2 \\end{bmatrix},  \\hat{y} = \\begin{bmatrix} y_1 \\\\ y_2 \\\\ \\vdots \\\\ y_n \\end{bmatrix}$$\n",
        "\n",
        "The element-wise MSE is taken to generate $n$ loss components, which are all summed together to generate a final loss."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6et--_-vP3GF"
      },
      "source": [
        "# 2. Recurrent Leaky Integrate-and-Fire Neurons\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IfX_Z2K9WzyP"
      },
      "source": [
        "Neurons in the brain have a ton of feedback connections. And so the SNN community have been exploring the dynamics of networks that feed output spikes back to the input. This is in addition to the recurrent dynamics of the membrane potential.\n",
        "\n",
        "There are a few ways to construct recurrent leaky integrate-and-fire (`RLeaky`) neurons in snnTorch. Refer to the [docs](https://snntorch.readthedocs.io/en/latest/snn.neurons_rleaky.html) for an exhaustive description of the neuron's hyperparameters. Let's see a few examples."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zr84Wt2WXbuF"
      },
      "source": [
        "## 2.1 RLIF Neurons with 1-to-1 connections\n",
        "\n",
        "<center>\n",
        "<img src='https://github.com/jeshraghian/snntorch/blob/master/docs/_static/img/examples/regression2/reg2-1.jpg?raw=true' width=\"600\">\n",
        "</center>\n",
        "\n",
        "\n",
        "This assumes each neuron feeds back its output spikes into itself, and only itself. There are no cross-coupled connections between neurons in the same layer."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "k2GJD63PXwef"
      },
      "outputs": [],
      "source": [
        "beta = 0.9 # membrane potential decay rate\n",
        "num_steps = 10 # 10 time steps\n",
        "\n",
        "rlif = snn.RLeaky(beta=beta, all_to_all=False) # initialize RLeaky Neuron\n",
        "spk, mem = rlif.init_rleaky() # initialize state variables\n",
        "x = torch.rand(1) # generate random input\n",
        "\n",
        "spk_recording = []\n",
        "mem_recording = []\n",
        "\n",
        "# run simulation\n",
        "for step in range(num_steps):\n",
        "  spk, mem = rlif(x, spk, mem)\n",
        "  spk_recording.append(spk)\n",
        "  mem_recording.append(mem)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MKPX_hpXaIQQ"
      },
      "source": [
        "By default, `V` is a learnable parameter that initializes to $1$ and will be updated during the training process. If you wish to disable learning, or use your own initialization variables, then you may do so as follows:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "DW-4IEw5abMV"
      },
      "outputs": [],
      "source": [
        "rlif = snn.RLeaky(beta=beta, all_to_all=False, learn_recurrent=False) # disable learning of recurrent connection\n",
        "rlif.V = torch.rand(1) # set this to layer size\n",
        "print(f\"The recurrent weight is: {rlif.V.item()}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "G2zFUtdsbFsl"
      },
      "source": [
        "## 2.2 RLIF Neurons with all-to-all connections\n",
        "\n",
        "### 2.2.1 Linear feedback \n",
        "\n",
        "<center>\n",
        "<img src='https://github.com/jeshraghian/snntorch/blob/master/docs/_static/img/examples/regression2/reg2-2.jpg?raw=true' width=\"600\">\n",
        "</center>\n",
        "\n",
        "By default, `RLeaky` assumes feedback connections where all spikes from a given layer are first weighted by a feedback layer before being passed to the input of all neurons. This introduces more parameters, but it is thought this helps with learning time-varying features in data."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "QTETc6HTbuLn"
      },
      "outputs": [],
      "source": [
        "beta = 0.9 # membrane potential decay rate\n",
        "num_steps = 10 # 10 time steps\n",
        "\n",
        "rlif = snn.RLeaky(beta=beta, linear_features=10)  # initialize RLeaky Neuron\n",
        "spk, mem = rlif.init_rleaky() # initialize state variables\n",
        "x = torch.rand(10) # generate random input\n",
        "\n",
        "spk_recording = []\n",
        "mem_recording = []\n",
        "\n",
        "# run simulation\n",
        "for step in range(num_steps):\n",
        "  spk, mem = rlif(x, spk, mem)\n",
        "  spk_recording.append(spk)\n",
        "  mem_recording.append(mem)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "elPt8Sf9b8UC"
      },
      "source": [
        "You can disable learning in the feedback layer with `learn_recurrent=False`. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Zifpg94-eVzS"
      },
      "source": [
        "### 2.2.2 Convolutional feedback\n",
        "If you are using a convolutional layer, this will throw an error because it does not make sense for the output spikes (3-dimensional) to be projected into 1-dimension by a `nn.Linear` feedback layer.\n",
        "\n",
        "To address this, you must specify that you are using a convolutional feedback layer:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "NwXKpKitdfmp"
      },
      "outputs": [],
      "source": [
        "beta = 0.9 # membrane potential decay rate\n",
        "num_steps = 10 # 10 time steps\n",
        "\n",
        "rlif = snn.RLeaky(beta=beta, conv2d_channels=3, kernel_size=(5,5))  # initialize RLeaky Neuron\n",
        "spk, mem = rlif.init_rleaky() # initialize state variables\n",
        "x = torch.rand(3, 32, 32) # generate random 3D input\n",
        "\n",
        "spk_recording = []\n",
        "mem_recording = []\n",
        "\n",
        "# run simulation\n",
        "for step in range(num_steps):\n",
        "  spk, mem = rlif(x, spk, mem)\n",
        "  spk_recording.append(spk)\n",
        "  mem_recording.append(mem)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rW_A7mq2dltO"
      },
      "source": [
        "To ensure the output spike dimension matches the input dimensions, padding is automatically applied.\n",
        "\n",
        "If you have exotically shaped data, you will need to construct your own feedback layers manually."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vmKuobrbP3GG"
      },
      "source": [
        "# 3. Construct Model\n",
        "\n",
        "Let's train a couple of models using `RLeaky` layers. For speed, we will train a model with linear feedback."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "UF105-e_P3GG"
      },
      "outputs": [],
      "source": [
        "class Net(torch.nn.Module):\n",
        "    \"\"\"Simple spiking neural network in snntorch.\"\"\"\n",
        "\n",
        "    def __init__(self, timesteps, hidden, beta):\n",
        "        super().__init__()\n",
        "        \n",
        "        self.timesteps = timesteps\n",
        "        self.hidden = hidden\n",
        "        self.beta = beta\n",
        "\n",
        "        # layer 1\n",
        "        self.fc1 = torch.nn.Linear(in_features=784, out_features=self.hidden)\n",
        "        self.rlif1 = snn.RLeaky(beta=self.beta, linear_features=self.hidden)\n",
        "\n",
        "        # layer 2\n",
        "        self.fc2 = torch.nn.Linear(in_features=self.hidden, out_features=10)\n",
        "        self.rlif2 = snn.RLeaky(beta=self.beta, linear_features=10)\n",
        "\n",
        "    def forward(self, x):\n",
        "        \"\"\"Forward pass for several time steps.\"\"\"\n",
        "\n",
        "        # Initalize membrane potential\n",
        "        spk1, mem1 = self.rlif1.init_rleaky()\n",
        "        spk2, mem2 = self.rlif2.init_rleaky()\n",
        "\n",
        "        # Empty lists to record outputs\n",
        "        spk_recording = []\n",
        "\n",
        "        for step in range(self.timesteps):\n",
        "            spk1, mem1 = self.rlif1(self.fc1(x), spk1, mem1)\n",
        "            spk2, mem2 = self.rlif2(self.fc2(spk1), spk2, mem2)\n",
        "            spk_recording.append(spk2)\n",
        "\n",
        "        return torch.stack(spk_recording)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6NvA9f4tP3GG"
      },
      "source": [
        "Instantiate the network below:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "EDV1AW5sP3GG"
      },
      "outputs": [],
      "source": [
        "hidden = 128\n",
        "device = torch.device(\"cuda\") if torch.cuda.is_available() else torch.device(\"cpu\")\n",
        "model = Net(timesteps=num_steps, hidden=hidden, beta=0.9).to(device)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "P8ziFcQNP3GG"
      },
      "source": [
        "# 4. Construct Training Loop\n",
        "\n",
        "## 4.1 Mean Square Error Loss in `snntorch.functional`\n",
        "\n",
        "From `snntorch.functional`, we call `mse_count_loss` to set the target neuron to fire 80% of the time, and incorrect neurons to fire 20% of the time. What it took 10 paragraphs to explain is achieved in one line of code:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "QN-ESMMKgcmB"
      },
      "outputs": [],
      "source": [
        "loss_function = SF.mse_count_loss(correct_rate=0.8, incorrect_rate=0.2)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3KkMK-pHhmpp"
      },
      "source": [
        "## 4.2 DataLoader\n",
        "\n",
        "Dataloader boilerplate. Let's just do MNIST, and testing this on temporal data is an exercise left to the reader/coder. "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "DM8AHNKAhu_b"
      },
      "outputs": [],
      "source": [
        "batch_size = 128\n",
        "data_path='tmp/data/mnist'\n",
        "\n",
        "# Define a transform\n",
        "transform = transforms.Compose([\n",
        "            transforms.Resize((28, 28)),\n",
        "            transforms.Grayscale(),\n",
        "            transforms.ToTensor(),\n",
        "            transforms.Normalize((0,), (1,))])\n",
        "\n",
        "mnist_train = datasets.MNIST(data_path, train=True, download=True, transform=transform)\n",
        "mnist_test = datasets.MNIST(data_path, train=False, download=True, transform=transform)\n",
        "\n",
        "# Create DataLoaders\n",
        "train_loader = DataLoader(mnist_train, batch_size=batch_size, shuffle=True, drop_last=True)\n",
        "test_loader = DataLoader(mnist_test, batch_size=batch_size, shuffle=True, drop_last=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "l0fJbPwolWo5"
      },
      "source": [
        "## 4.3 Train Network"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "IQDb0IbtP3GG"
      },
      "outputs": [],
      "source": [
        "num_epochs = 5\n",
        "optimizer = torch.optim.Adam(params=model.parameters(), lr=1e-3)\n",
        "loss_hist = []\n",
        "\n",
        "with tqdm.trange(num_epochs) as pbar:\n",
        "    for _ in pbar:\n",
        "        train_batch = iter(train_loader)\n",
        "        minibatch_counter = 0\n",
        "        loss_epoch = []\n",
        "\n",
        "        for feature, label in train_batch:\n",
        "            feature = feature.to(device)\n",
        "            label = label.to(device)\n",
        "\n",
        "            spk = model(feature.flatten(1)) # forward-pass\n",
        "            loss_val = loss_function(spk, label) # apply loss\n",
        "            optimizer.zero_grad() # zero out gradients\n",
        "            loss_val.backward() # calculate gradients\n",
        "            optimizer.step() # update weights\n",
        "\n",
        "            loss_hist.append(loss_val.item())\n",
        "            minibatch_counter += 1\n",
        "\n",
        "            avg_batch_loss = sum(loss_hist) / minibatch_counter\n",
        "            pbar.set_postfix(loss=\"%.3e\" % avg_batch_loss)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oYWvNlrMP3GG"
      },
      "source": [
        "# 5. Evaluation"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Y1FZVWmKiSlG"
      },
      "outputs": [],
      "source": [
        "test_batch = iter(test_loader)\n",
        "minibatch_counter = 0\n",
        "loss_epoch = []\n",
        "\n",
        "model.eval()\n",
        "with torch.no_grad():\n",
        "  total = 0\n",
        "  acc = 0\n",
        "  for feature, label in test_batch:\n",
        "      feature = feature.to(device)\n",
        "      label = label.to(device)\n",
        "\n",
        "      spk = model(feature.flatten(1)) # forward-pass\n",
        "      acc += SF.accuracy_rate(spk, label) * spk.size(1)\n",
        "      total += spk.size(1)\n",
        "\n",
        "print(f\"The total accuracy on the test set is: {(acc/total) * 100:.2f}%\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JBZcm69iP3GG"
      },
      "source": [
        "# 6. Alternative Loss Metric\n",
        "\n",
        "In the previous tutorial, we tested membrane potential learning. We can do the same here by setting the target neuron to reach a membrane potential greater than the firing threshold, and incorrect neurons to reach a membrane potential below the firing threshold:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "zOeuHY-lm4Z3"
      },
      "outputs": [],
      "source": [
        "loss_function = SF.mse_membrane_loss(on_target=1.05, off_target=0.2)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JIdk4LsRnBwE"
      },
      "source": [
        "In the above case, we are trying to get the correct neuron to constantly sit above the firing threshold.\n",
        "\n",
        "Try updating the network and the training loop to make this work. \n",
        "\n",
        "Hints:\n",
        "* You will need to return the output membrane potential instead of spikes.\n",
        "* Pass membrane potential to the loss function instead of spikes"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Cq_plL_wP3GH"
      },
      "source": [
        "# Conclusion\n",
        "\n",
        "The next regression tutorial will introduce spiking LSTMs to achieve precise spike time learning.\n",
        "\n",
        "If you like this project, please consider starring ⭐ the repo on GitHub as it is the easiest and best way to support it.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4iEHtimBP3GH"
      },
      "source": [
        "# Additional Resources\n",
        "* [Check out the snnTorch GitHub project here.](https://github.com/jeshraghian/snntorch)\n",
        "* More detail on nonlinear regression with SNNs can be found in our corresponding preprint here: [Henkes, A.; Eshraghian, J. K.; and Wessels, H.  “Spiking neural networks for nonlinear regression\", arXiv preprint arXiv:2210.03515, Oct. 2022.](https://arxiv.org/abs/2210.03515) "
      ]
    }
  ],
  "metadata": {
    "celltoolbar": "Raw Cell Format",
    "colab": {
      "collapsed_sections": [
        "9QXsrr6Mp5e_",
        "1EWDw3bip8Ie",
        "vFM8UV9CreIX",
        "xXkTAJ9ws1Y6",
        "OgkWg605tE1y",
        "OBt0WDzyujnk",
        "xC96eesMqYo-",
        "mszPTrYOluym",
        "VTHK-wAWV57B"
      ],
      "name": "snntorch_regression_2.ipynb",
      "provenance": []
    },
    "gpuClass": "standard",
    "kernelspec": {
      "display_name": "torch-gpu",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.15"
    },
    "vscode": {
      "interpreter": {
        "hash": "2a3056c17c3c31a88ffeb08a28ff32bf922ba3f6fa0343ca62cc95241e30c809"
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
